Overcomplete systems of wavelet and related local bases for adaptive signal representation and estimation

In this paper we will discuss the usefulness of overcomplete systems of basis functions, such as wavelets or localized sine and cosine functions, for the adaptive parsimonious representation and estimation of statistical signals which show an inhomogeneous behaviour over time. Typical examples can be found in electrical engineering, sound and speech processing, geophysics, biomedicine, etc. We illustrate this along two methods which aim at estimation of time-dependent autocovariance or spectrum of non-stationary processes: the ”Auto-SLEX” method of Ombao et al (1999) and the approach on ”locally stationary wavelet processes” by Nason et al (1998). Common to both is on one hand the inherent redundancy of the used overcomplete system which is the key to adaptation. On the other hand, to allow for rigorous modeling and statistical estimation, a control of this redundancy is needed which is the price for the flexibility of overcomplete systems in comparison with orthogonal ones. Two different though related possibilities of a controlled overcompleteness will be investigated here: on one hand collections of ortho-basis from which a certain ”best” element (best adapted to the signal) is to be searched for, on the other hand, translation–invariant (and hence overcomplete) representations for the autocovariance function of a possible non–stationary stochastic signal. The idea of using an overcomplete collection (i.e. a ”library”) of orthogonal basis functions to best represent the content of a given signal goes back to Coifman and Wickerhauser (1991). From a general, not necessarily statistical, point of view one tries to match the predominant signal features by a set of ”independent”, i.e. orthogonal, components. This so-called search of a ”Best Basis (BB)”, achieved by a computationally efficient and fast algorithm, has recently been extended to sparsely represent (and subsequently estimate) the characteristic quantities of stochastic signals, such as mean, autocovariance or spectrum. In its original form the BB search is performed over a library of either ”cosine packets” or ”wavelet packets”, i.e. a particularly time-localized Fourier basis or a particularly further frequency-localized wavelet basis. With both, an additional flexibility for a better adapted localized representation of the given signal in the time-frequency plane is achieved compared to a pure Fourier (or sine or cosine) or a pure wavelet expansion. This applies foremost to signals with features simultaneously localised in frequency and in time. The search over the given library of bases elements is performed by maximisation of a given convex cost function (”neg-entropy” criterion) for which several suggestions exist. We will now discuss application of this principle to two special sets of basis functions and will be more detailed in presenting the first method as it is the more recent one.